As individuals and organizations continue to rapidly accumulate large collections of image content, they increasingly will require systems and methods for organizing and browsing the image content in their collections. Many systems allow a user to manually classify images and other digital content by their association with a particular event or subject matter and to segment this digital content based on these classifications. Manual classification systems, however, tend to be time consuming and unwieldy, especially as the size of the digital content collection grows. Some systems are configured to automatically segment digital content, such as images, based on color, shape, or texture features. These automatic segmentation systems, however, often tend to misclassify the digital content because of the inherent inaccuracies associated with classification based on color, shape, or texture features.
In some content-based image retrieval approaches, low level visual features are used to group images into meaningful categories that, in turn, are used to generate indices for a database containing the images. In accordance with these approaches, images are represented by low level features, such as color, texture, shape, and layout. The features of a query image may be used to retrieve images in the databases that have similar features. In general, the results of automatic categorization and indexing of images improve when the features that are used to categorize and index the images more accurately capture the target aspects of the content of the images.
Recently, efforts have been made to detect and classify aspects (e.g., faces and eyes) of human subjects. For example, in one approach, human faces in digital images are organized into clusters. In accordance with this approach, a face image is used to form a first cluster. A face recognizer generates similarity scores based on comparisons of an unassigned face image to each face image in any existing cluster. If a similarity score for the unassigned face image is above a threshold, the unassigned face image is added to the cluster corresponding to the highest similarity score. If the similarity scores for the unassigned face image is below the threshold, the unassigned face image is used to form a new cluster. The process is repeated for each unassigned face image. For each cluster, all the face images contained in the cluster are displayed in a cluster review screen. If a face image does not belong to a particular cluster, the user may delete it from the cluster or reassign to another cluster. If two clusters of faces belong together, the user can merge the clusters.
In a semi-automatic face clustering approach, a face detector is used to automatically extract faces from photos. A face recognizer is used to sort faces by their similarity to a chosen model. In this process, when one or more faces are associated with a person, a model representing those faces is created and unlabeled faces that are sorted in order of their similarity to that model are displayed. The user can select several faces and assign them to the appropriate person with a drag-and-drop interaction technique. The model is updated to incorporate the newly identified faces and the unlabeled faces are resorted by their similarity to the new model. The sorted faces are presented as candidates within a user interface that allows the user to label the faces.
The face models that are built using existing face-based image clustering approaches typically do not accurately represent the faces of the persons they represent. As a result, faces that are ranked closest to a particular face model oftentimes do not correspond to the person that is represented by the model. In particular, if one image is mistakenly included into one cluster, then other images that are similar to this image also will be included into this cluster. In this way, each false alarm may propagate to generate more false alarms. What are needed are face-based image clustering systems and methods that are capable of building more accurate and more robust face models.